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16.  Abstract 

The  Systems  Research  and  Development  Service  (SRDS)  asked  the  Federal  Aviation 
Administration  (FAA)  Technical  Center  to  develop  and  demonstrate  the  capability  of 
automatic  flight  plan  filing  by  computerized  word  recognition.  To  accomplish  a 
prototype  capability,  it  was  necessary  to  begin  gathering  human  factors  data  to  help 
determine  the  correct  technical  approach  and  design  concept.  The  tests  reported 
herein  centered  on  two  questions:  Would  pilots  use  this  method  of  flight  plan 
entry?  Which  of  the  three  protocols  simulated  do  pilots  prefer  most?  Because  of 
the  limitations  inherent  in  this  expertaent,  the  answers  to  these  questions  are 
tentative  at  this  time.  Given  the  assumption  that  the  actual  utterance  recognition 
(device  (URD)  performs  at  a  level  comparable  to  the  simulation  used  in  this  experiment, 
it  is  entirely  reasonable  to  conclude  that  general  aviation  pilots  would  (1)  elect 
to  file  flight  plans  by  computerized  word  recognition  using  a  real  computer  and 
(2)  prefer  "no  prompt"  or  a  "word  echo-back  prompt"  over  the  "cue-tone  prompt." 
[The  results  show  that  the  preferred  protocol  was  the  "no  prompt"  method. 
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INTRODUCTION 


PURPOSE. 

The  purpose  of  the  experiment  documented 
in  this  report  is  to  suggest  tentstive 
answers  to  the  following  questions: 

1.  Would  pilots  use  s  method  of 
direct  flight  plsn  entry  by  computerized 
word  recognition? 

2.  Which  one  of  three  protocols 
simulated  do  pilots  prefer  most? 

Although  the  answers  to  these  questions 
as  obtained  through  this  experiment  are 
tentative  and  preliminary  in  nature, 
they  may  serve  as  a  basis  for  more 
comprehensive  testing  and  subsequent 
system  design. 

BACKGROUND. 

The  Mass  Weather  Dissemination  System 
Exploratory  Model  located  at  the  Federal 
Aviation  Administration  (FAA)  Technical 
Center  is  a  developmental  effort 
sponsored  by  Systems  Research  and 
Development  Service  (SRDS).  One  of  the 
recent  objectives  of  this  project  is  to 
develop  a  prototype  model  with  the 
capability  of  direct  flight  plan  filing 
by  computerized  word  recognition.  A 
major  component  of  the  system  is  an 
utterance  recognition  device  (URD)  used 
for  the  recognition  of  human  speech 
over  telephone  lines.  It  is  the  only 
known  commercially  available  word 
recognition  machine  designed  for  use 
on  a  switched  telecommunications 
system.  It  is  an  eight-channel  discrete 
utterance  recognition  machine  designed 
to  recognize  words  for  which  it  has 
been  programed.  The  URD  is  currently 
programed  to  recognize  25  words. 

SRDS  has  asked  the  Technical  Center  to 
develop  and  demonstrate  the  capability 
of  automatic  flight  plan  filing  by  voice 
recognition.  Improvements  to  the  URD, 
which  are  now  underway,  include  modifi¬ 
cations  to  the  software  and  hardware  and 


expansion  of  the  present  vocabulary  to 
include  the  international  phonetic 
alphabet  and  additional  command  words. 

MAJOR  LIMITATIONS  OF  THE  EXPERIMENT. 

To  enable  the  reader  to  judge  the 
validity  of  the  conclusions  drawn  from 
the  data  and  the  general  worth  of  the 
experiment,  it  is  important  to  point  out 
the  following  major  limitations.  First, 
an  actual  voice  recognition  machine  was 
not  used.  The  machine  recognition  and 
voice  was  simulated  by  a  person  imita¬ 
ting  the  machine  (see  figure  1).  This 
was  unavoidable  as  the  expanded  vocabu¬ 
lary  (phonetic  alphabet  and  additional 
command  words)  was  not  yet  available. 
Second,  the  pilots  who  participated  in 
this  experiment  were  professional 
Federal  Aviation  Administration  (FAA) 
pilots  and  other  Technical  Center 
employees  and,  as  a  group,  were  not 
typical  of  the  general  aviation 
community.  However,  it  was  felt  that 
their  evaluation  of  the  concept  would  be 
of  distinct  value  in  light  of  their 
substantial  aviation  knowledge  and 
experience. 


EXPERIMENTAL  PROCEDURE 


The  subjects  were  volunteers  from  the 
pilot  population  at  the  Technical 
Center.  They  were  all  FAA  employees 
with  various  levels  of  experience  from  a 
private  pilot  with  100  hours  total 
flying  time  to  an  airline  transport 
pilot  with  over  23,000  hours.  The 
subjects  were  tested  individually.  Most 
were  not  familiar  with  the  concept  of 
direct  flight  plan  filing  by  compu¬ 
terized  word  recognition.  Three 
different  flight  plans  were  used 
on  which  five  different  tests  were 
conducted.  The  flight  plans  were 
typical  instrument  flight  rules  (IFR), 
low  altitude,  high  performance  aircraft. 
The  plans  were  nearly  identical  except 
for  the  route  of  flight  which  was 
changed  slightly  in  each  case  to 
maintain  interest  and  to  provoke  ideas 
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and  comments.  The  computer  voice 
reeponae  was  simulated  by  a  test  team 
member  using  a  telephone  in  a  different 
room.  To  maintain  a  high  level  of 
realism  throughout  the  experiment,  the 
tester  or  simulated  "computer"  responded 
to  the  pilot's  input  as  an  actual 
computer  might.  This  included  the 
insertion  of  typical  errors  for  the 
pilot  to  correct.  A  second  team 
member  remained  with  the  test  subject, 
monitored  the  telephone  dialog  between 
pilot  and  "computer,"  and  controlled 
the  test. 

The  subject  was  presented  with  a  sheet 
containing  the  three  flight  plans  in 
order  of  testing.  It  was  felt  that 
having  all  the  information  on  one  work 
sheet  would  be  an  aid  to  the  subject  in 
making  later  comparisons. 

Test  instructions  were  explained  while 
referring  to  the  flight  plan  sample 
sheet.  This  also  gave  the  subjects  a 
chance  to  familiarize  themselves  with 
the  material  before  testing.  Since 
method  of  entry  differed  with  each 
flight  plan,  the  initial  explanation 
was  centered  exclusively  on  flight  plan 
A  to  minimize  confusion.  The  method  of 
testing  and  the  overall  objectives  were 
briefly  explained.  Flight  plan  informa¬ 
tion  was  entered  using  the  phonetic 
alphabet,  and  the  "computer"  verified 
the  entries  by  a  readback.  It  was 
explained  that  the  computer  was 
programed  to  recognize  the  phonetic 
alphabet,  the  numbers  zero  through  nine, 
and  certain  command  words  (figure  2). 
Thus,  all  identifiers,  names,  etc., 
were  required  to  be  spelled  out.  For 
example,  the  aircraft ' identifier  C310/U 
was  entered  as  "Charlie  Three  One  Zero 
Slant  Uniform."  The  word  slant  was  a 
command  word  the  "computer"  recognized 
and  thus  did  not  have  to  be  spelled  out. 
It  separated  the  aircreft  from  the 
special  equipment  code.  Omitting  it 
would  generate  an  error.  Likewise,  in 
the  "route  of  flight"  each  route  segment 
was  separated  by  a  slant  and  again  was 


entered  to  avoid  an  error.  The  route  of 
flight  was  entered  as  "Victor  One  Eight 
Four  Slant  Mike  X-ray  Echo  Slant...." 

All  times  were  given  using  a  24-hour 
clock  and  were  assume^  to  be  Greenwich. 
Elapsed  times  were  given  on  a  24-hour 
basis  also.  Thus,  4  hours  15  minutes 
fuel  on  board  (or  estimated  time  en 
route)  was  given  as  0415  spoken 
"Zero  Four  One  Five  (leading  zero  is 
optional)."  Cruising  altitude 
8,000  feet  was  entered  as  "Eight  Zero 
Zero  Zero."  A  visual  flight  rules  (VFR) 
altitude  of  6,500  feet  was  "Six  Five 
Zero  Zero." 

All  information  was  to  be  entered  in  the 
same  manner.  If  a  word  that  was  not  in 
the  "computer's"  vocabulary  was  inadver¬ 
tantly  entered,  the  "computer"  read  back 
a  word  it  "thought"  most  closely 
resembled  the  word  entered.  For 
example,  the  phonetically  correct  INDIA 
might  be  mistakenly  entered  as  ITEM. 
Since  ITEM  was  not  in  the  "computer's" 
vocabulary  it  might  read  back  SEVEN  or 
DELTA.  The  pilot  must  detect  this  error 
and  enter  the  appropriate  correction. 
It  was  explained  to  the  subject  that  the 
methods  of  correction  would  vary  among 
the  flight  plans  tested,  and  one  of  our 
objectives  was  to  select  the  best 
method.  The  pilots  were  told  that  all 
entries  had  to  be  clear  and  distinct  or 
the  "computer"  would  not  be  able  to 
recognize  a  word  even  if  it  was  correct. 
It  was  stressed  that  an  actual  operating 
system  such  as  this  would  not  be  very 
tolerant  of  loud  background  noise  or  any 
extraneous  noises  or  extra  syllables 
that  normal  conversat ional  speech  might 
generate.  An  error  could  be  caused  by 
slurring  the  entry,  clearing  the  throat, 
or  mumbling.  The  person  simulating  the 
URD  was  especially  watchful  for  these 
factors  during  the  test  and  attempted  to 
accurately  duplicate  what  the  actual 
computer's  response  might  be  to  these 
extraneous  inputs.  He  would  either  read 
back  something  different  or  say  "PLEASE 
REPEAT,"  after  which  the  subject  would 
have  to  reenter  the  correct  word. 


VOCABULARY 


PHONETIC  ALPHABET 

NUMBERS 

COMMAND 

WORDS 

ALPHA 

NOVEMBER 

ONE 

AFFIRMATIVE 

FLIGHT  LEVEL 

BRAVO 

OSCAR 

TWO 

YES 

DIRECT 

CHARLIE 

PAPA 

THREE 

NEGATIVE 

SID 

DELTA 

QUEBEC 

FOUR 

NO 

STAR 

ECHO 

ROMEO 

FIVE 

EAST 

INTERCEPT 

FOXTROT 

SIERRA 

SIX 

WEST 

RADIAL 

GOLF 

TANGO 

SEVEN 

NORTH 

JET 

HOTEL 

UNIFORM 

EIGHT 

SOUTH 

MILES 

INDIA  • 

VICTOR 

NINE 

LOCAL 

POINT 

JULIETT 

WHISKEY 

NINER 

AMEND 

SLANT 

KILO 

X-RAY 

ZERO 

CLOSE 

LAST 

LIMA 

YANKEE 

MIKE 

ZULU 

FIGURE  2.  SIMULATED  TEST  VOCABULARY 
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TEST  ONE ,  COB-TOME  (FLIGHT  PLAN  A). 

Flight  plan  A  utilized  in  teat  one  is 
shown  in  figure  3.  After  dialing  up 
the  "computer,"  communication  was 
established  after  two  rings,  and  a  short 
introduction  was  given  as  follows: 
"HELLO,  YOU  ARE  CONNECTED  TO  AUTOMATIC 
FLIGHT  PLAN  FILING  VIA  COMPUTER  WORD 
RECOGNITION.  PLEASE  SPEAK  EACH  WORD 
CLEARLY  AND  DISTINCTLY  AND  WAIT  FOR  THE 
PROMPT  BEFORE  SPEAKING  EACH  WORD.  A 
PAUSE  OF  5  SECONDS  WILL  TERMINATE  EACH 
LINE  OF  INFORMATION.  ONLY  VFR  AND  IFR 
FLIGHT  PLANS  MAY  BE  ENTERED.  IS  THIS 
A  VFR  FLIGHT  PLAN?" 

The  pilot  was  instructed  to  respond 
"No,"  as  an  appropriate  answer  for  this 
particular  test.  It  was  explained  that 
YES,  NO,  AFFIRMATIVE,  NEGATIVE  were  all 
recognized  command  words,  and  he  could 
use  whichever  was  most  natural  for  him 
throughout  the  test.  This  introduction 
was  repeated  for  every  teat. 

The  "computer"  then  initiated  the  first 
line  of  entry  by  directing  him  to 
ENTER  AIRCRAFT  IDENTIFICATION.  This 
command  was  immediately  followed  by  a 
beep  tone  which  was  the  pilot's  cue  to 
enter  the  first  character  N  or  November. 
At  this  time  it  was  stressed  that  this 
first  test  was  the  beep  cue-tone  method, 
and  it  was  important  to  wait  for  the 
tone  before  making  any  entries.  After 
about  a  1-  to  2-second  pause,  another 
tone  is  heard  which  was  the  cue  to  enter 
the  next  element.  The  test  conductor 
demonstrated  the  correct  method  of 
entering  the  complete  aircraft  identifi¬ 
cation  block,  including  the  beep  tones 
and  proper  cadence.  A  5-second  pause 
indicated  to  the  "computer"  that  the 
entry  was  complete  and  the  "computer" 
initiated  a  phonetic  readback  of  the 
information  to  the  pilot.  This  was 
also  demonstrated.  If  an  error  was 
detected,  the  pilot  was  to  wait  until 
the  "computer"  was  finished  with  the 
entire  readback,  and  then  say  "no"  or 
"negative."  There  was  an  immediate  beep 
tone  after  which  the  pilot  reentered 


the  data  as  before,  this  time  correcting 
the  error.  Once  again,  after  the 
5-Becond  pause,  the  information  was  read 
back.  If  a  new  error  was  detected, 
the  correction  process  was  repeated.  If 
the  readback  was  acceptable,  3  seconds 
of  silence  initiated  the  next  line  of 
entry.  In  all  cases  a  correction 
required  the  reentry  of  all  the  informa¬ 
tion  in  that  line.  If  a  pilot  inadver¬ 
tantly  paused  for  5  seconds  before 
finishing  a  line,  and  the  "computer" 
had  initiated  the  next  line  of  entry,  he 
could  say  the  command  word  LAST,  and 
the  "computer"  would  reinitiate  the 
previous  line  for  reentry.  Changes  were 
made  in  the  same  manner,  as  the  pilot 
had  the  capability  to  back  his  way 
through  the  flight  plan. 

A  partial  demonstration  of  entry  for 
"route  of  flight"  was  given.  Since 
the  remarks  section  of  flight  plan  A  was 
blank,  a  5-second  pause  after  the  beep 
was  all  that  was  required  for  the 
computer  to  move  to  the  next  line. 

After  remarks,  beginning  with  item  11, 
"fuel  on  board,"  all  entries  were  tape 
recorded  in  the  conventional  manner  and 
stored  for  future  reference,  such  as 
for  search  and  rescue  use.  The  five 
remaining  entries  would  not  be  entered 
by  computerized  word  recognition.  The 
user  could  enter  the  data  in  any  manner 
he  chose  as  it  would  not  be  processed  by 
the  "computer."  While  important  for 
search  and  rescue  or  other  needs,  this 
information  is  not  essential  for 
computer  processing  of  his  flight 
plan  into  the  National  Airspace 
System  (NAS). 

The  ealaphotta  transaction  was  completed 
by  the  closing  statement  by  the 
"computer"  as  follows:  "YOUR  FLIGHT  PLAN 
HAS  BEEN  ACCEPTED  BY  THE  COMPUTER. 
THANK  YOU.  HAVE  A  GOOD  FLIGHT." 

The  pilots  were  reminded  to  speak  slowly 
and  distinctly,  to  wait  for  the  cue-tone 
before  entering  anything,  and  to  begin 
the  test. 
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1.  AIRCRAFT  IDENTIFICATION:  N123B 

2.  AIRCRAFT  TYPE  &  SPECIAL  EQUIPMENT:  C3I0/U 

3.  DESTINATION:  PIT 

4.  ESTIMATED  TIME  EN  ROUTE:  0215 

5.  AIRCRAFT  TRUE  AIRSPEED:  180 

6.  DEPARTURE  POINT:  ACY 

7.  DEPARTURE  TIME:  1600 

8.  CRUISING  ALTITUDE:  8,000 

9.  ROUTE  OF  FLIGHT:  V184/MXE/V474/THS/V12S/JST 

10 .  REMARKS : 

11.  FUEL  ON  BOARD:  0400 

12.  ALTERNATE  AIRPORT:  YNG 

13.  PILOT’S  NAME,  ADDRESS  &  TELEPHONE  NO.,  AND  AIRCRAFT  HOME  BASE: 

14.  NUMBER  ABOARD:  2 

15 .  COLOR  OF  AIRCRAFT :  WHITE/RED 


FIGURE  3.  FLIGHT  PLAN  A 
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TEST  TWO,  WORD  ECHO-BACK 
(FLIGHT  PLAN  B). 

Flight  plan  B  utilized  in  teat  two  ia 
ahown  in  figure  4.  All  entriea  were 
at'ill  made  phonetically.  The  main 
difference  waa  the  abaence  of  the  beep 
cue-tone,  which  was  replaced  by  an 
echo-back  of  the  word  entered.  The 
"computer"  initiates  the  first  line  of 
the  flight  plan  by  saying  ENTER  AIRCRAFT 
IDENTIFICATION.  There  is  no  beep, 
and  the  pilot  enters  the  first  word, 
"November."  The  computer  reads  back 
NOVEMBER.  This  verifies  the  entry  for 
the  pilot  and  is  a  cue  for  him  to 
proceed.  After  a  1-second  pause  he 
says  "One,"  and  if  the  computer  again 
recognizes  the  word,  it  echoes  back  ONE, 
and  so  forth.  With  this  method  of 
prompting,  no  readback  was  given  after 
the  5-second  terminating  pause.  If 
the  vocabulary  word  echoed  back  was 
incorrect  (i.e.,  the  word  had  been 
misrecognized) ,  the  command  word  "NO" 
or  "NEGATIVE"  was  spoken,  and  after 
1  aacond,  the  correct  word  was  reentered 
and  the  correction  echoed  back.  This 
method  enabled  the  user  to  get  immediate 
confirmation  of  the  word  being  entered, 
and  enabled  him  to  make  an  immediate 
correction.  It  also  eliminated  the  need 
for  a  complete  readback  of  the  line 
entered. 

A  demonstration  of  proper  entry  was 
performed  using  the  aircraft  identifi¬ 
cation  block  as  "November .. .NOVEMBER. . . 
One. . .ONE. . .Two. . .TWO. . .Three.  .  .etc." 

The  technique  for  correcting  was  demon¬ 
strated  as  "November .. .NOVEMBER. . .One .. . 
NINE... No,  One. . .ONE. . .etc." 

The  command  word  DIRECT  was  introduced. 
Since  this  word  is  in  the  "computer's" 
vocabulary,  it  is  not  entered 
phonetically.  Its  use  in  the  route  of 
flight  was  demonstrated  as  "Direct... 
DIRECT.  .  .Slant .  .  .SLANT.  .  .Victor.  .  . 
VICTOR. . .etc." 

The  command  words  NO,  SID,  and  STAR 
were  also  introduced  into  the  remarks 


section.  As  in  the  other  entries,  they 
are  pronounced  individually  with  an 
echo-back  after  each  one. 

All  entries  after  the  remarks  section 
are  tape  recorded  for  local  storage 
as  in  the  first  test.  The  computer 
initiates  the  line  by  saying  ENTER 
FUEL  ON  BOARD  and  the  pilot  can  enter 
the  information  as  normally  spoken. 
He  will  get  no  correction  or  readback. 
The  computer  only  waits  for  the  5-second 
pause  to  terminate  the  line  and  proceed 
with  the  next. 

TEST  THREE,  NO  PROMPT  (FLIGHT  PLAN  C) . 

Flight  plan  C  utilized  in  test  three 
is  shown  in  figure  5.  All  entries  are 
still  made  phonetically.  The  main 
difference  is  the  entire  line  of 
information  is  entered  without  a 
cue-tone  or  echo-back.  There  is  a 
2-  to  3-second  pause  between  words.  At 
the  end  of  5  seconds  the  "computer" 
verifies  the  information  by  reading 
back  the  entire  line.  If  the  readback 
contains  an  error,  the  subject  first 
waits  for  the  computer  to  finish,  then 
says  "No"  or  "Negative,"  and  immediately 
reenters  all  the  data  with  the 
correction.  If  satisfactory,  the 
computer  moves  to  the  next  line  after 
5  seconds.  When  making  the  entries, 
a  pause  of  at  least  2  to  3  seconds 
between  words  is  required.  Whereas  in 
previous  tests  the  computer  provided 
this  separation  through  the  cue-tone  or 
word  echo-back,  the  subject  now  had 
this  responsibility;  and  this  point 
was  emphasized.  A  demonstration 
was  performed  using  the  aircraft 
identification.  The  test  conductor 
stressed  the  proper  cadence  to  the 
subject.  Most  test  subjects  attained 
the  desired  cadence  throughout  the  test. 

A  standard  instrument  departure  (SID) 
and  a  standard  terminal  arrival  route 
(STAR)  were  added  to  the  route  of  flight 
in  this  test.  The  SID  is  a  "denim  one 
departure"  with  a  "sea  isle  transition," 
and  normally  pronounced  that  way  by  the 
pilot  when  filing.  Computer  entry 
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AIRCRAFT  IDENTIFICATION:  N123B 


2.  AIRCRAFT  TYPE  &  SPECIAL  EQUIPMENT:  C310/U 

3.  DESTINATION:  ISP 

4.  ESTIMATED  TIME  EN  ROUTE:  0215 

5.  AIRCRAFT  TRUE  AIRSPEED:  180 

6.  DEPARTURE  POINT:  PIT 

7.  DEPARTURE  TIME:  2000 

8.  CRUISING  ALTITUDE:  9,000 

9.  ROUTE  OF  FLIGHT:  DIRECT/AGC/V12/JST/V35/PSB/V6/BWZ/V232/LGA 

10.  REMARKS:  NO  SID  STAR 

11.  FUEL  ON  BOARD:  0400 

12.  ALTERNATE  AIRPORT:  ACY 

13.  PILOT'S  NAME,  ADDRESS  &  TELEPHONE  NO.,  AND  AIRCRAFT  HOME  BASE: 

14.  NUMBER  ABOARD:  4 

15.  COLOR  OF  AIRCRAFT:  WHITE/RED  I 


FIGURE  4.  FLIGHT  PLAN  B 


1.  AIRCRAFT  IDENTIFICATION:  N123B 

2.  AIRCRAFT  TYPE  &  SPECIAL  EQUIPMENT:  C310/U 

3.  DESTINATION:  ATL 

4.  ESTIMATED  TIME  EN  ROUTE:  0340 

3.  AIRCRAFT  TRUE  AIRSPEED:  180 

6.  DEPARTURE  POINT:  ISP 

7.  DEPARTURE  TIME:  2300 

8.  CRUISING  ALTITUDE:  8,000 

9.  ROUTE  OF  FLIGHT:  DENIM1 .SIE/V308/HEDGE/V16/LYH/V16S/PSK/ 

PSK.MACEY1 

10.  REMARKS: 

11.  FUEL  ON  BOARD:  0445 

12.  ALTERNATE  AIRPORT:  JAX 

13.  PILOT'S  NAME,  ADDRESS  &  TELEPHONE  NO.,  AND  AIRCRAFT  HOME  BASE: 

14.  NUMBER  ABOARD:  3 

15.  COLOR  OF  AIRCRAFT:  WHITE/RED 


FIGURE  5.  FLIGHT  PLAN  C 
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appear*  as  "DENIM1 . SEAISLE"  and  is 
entered  phonetically  as  "Delta  Echo 
November  India  Mike  One  Point  Sierra 
Echo  Alpha  India  Sierra  Lima  Echo." 
The  STAR  is  a  "macey  one  arrival," 
"pulaski  transition,"  and  normally 
pronounced  that  way  by  the  pilot  when 
filing.  Computer  entry  would  appear 
as  "PSK.MACEY1"  and  would  be  entered 
phonetically  as  "Papa  Sierra  Kilo  Point 
Mike  Alpha  Charlie  Echo  Yankee  One." 
POINT  is  a  new  command  word. 

A  change  in  the  method  of  entry  for  the 
route  of  flight  was  introduced.  The 
subject  entered  the  route  data  in  its 
entirety,  the  same  as  the  other  entries. 
The  "computer"  gave  the  readback  after 
5  seconds.  The  "computer"  paused  at  the 
slant  and  permitted  a  correction  of  the 
data  up  to  that  point  instead  of  reading 
back  the  entire  block  before  the  pilot 
had  an  opportunity  to  say  "No"  for  a 
correction.  The  "computer"  did  not  read 
back  the  word  SLANT  but  substituted  it 
with  a  pause  of  about  2  seconds  to 
allow  the  pilot  to  say  "No"  if  an  error 
had  been  detected.  This  method  allows 
correction  of  data  by  route  segments 
and  precludes  the  need  to  reenter  the 
entire  route  when  there  is  only  a 
single  mistake.  Only  the  small  segment 
containing  the  error  would  have  to  be 
entered.  A  correction  sequence  would 
appear  as  "V16/LYH/ . . . ; "  entered  as 
"Victor  One  Six  Slant  Lima  Yankee  Hotel 
Slant...;"  and  readback  with  an  error 
and  corrected  as  "VICTOR  ONE  SIX... LIMA 
TANGO  HOTEL .. . Negat ive ,  Lima  Yankee 

Hotel.. .LIMA  YANKEE  HOTEL _ "  Again, 

beginning  with  "fuel  on  board,"  all 
entries  were  made  as  in  the  previous 
tests. 

#  r 

After  completing  flight  plan  C,  the 
pilot  was  asked  to  select  the  preferred 
method  up  to  that  point. 

TEST  FOUR,  PARTIAL  INFORMATION 
REQUEST  (OPTIONAL  FLIGHT  PLAN)'. 

The  optional  flight  plan  utilised  in 
test  four  and  five  is  shown  in  figure  6. 


In  the  first  three  teats,  the  computer 
initiated  the  action  by  giving  a 
full  information  request  to  the  pilot 
each  time  data  was  to  be  entered.  In 
test  four  a  partial  request  was  tried, 
and  instead  of  the  computer  saying 
ENTER  AIRCRAFT  IDENTIFICATION,  it  said 
ENTER  BLOCK  ONE.  After  the  first 
line,  this  was  shortened  even  further  by 
omitting  the  word  ENTER  and  simply 
saying  BLOCK  TWO... BLOCK  THREE... etc. 

The  test  subject  had  the  option  of 
selecting  any  of  the  three  flight  plans 
to  reenter  for  testing  of  this  new 
feature.  All  other  test  procedures 
remained  the  same  for  the  flight  plan  he 
selected.  It  was  felt  that  selecting 
his  own  plan  for  this  test  would  give 
him  the  opportunity  to  review  any 
previous  method  before  making  a  positive 
selection. 

TEST  FIVE,  VISUAL  FLIGHT  RULES 
(VFR)  FLIGHT  PLAN  ENTRY  (OPTIONAL 
FLIGHT  PLANTS 

In  the  VFR  flight  plan  test,  only  the 
first  four  lines  of  information  are 
processed  through  word  recognition. 
Lines  5  through  15  are  recorded  for 
future  reference  as  needed.  As  in 
the  other  flight  plan  tests,  this 
recorded  information  can  be  entered  in 
any  manner.  The  subject  again  had 
the  choice  of  selecting  any  of  three 
sample  flight  plans  to  enter  and  the 
choice  of  full  information  request  or 
the  block  method  of  entry. 

Upon  completion  of  the  fifth  test,  the 
subject  filled  out  the  questionnaire. 
Comments  were  encouraged  not  only  in  the 
special  comments  section,  but  also 
anywhere  in  the  questionnaire  where  they 
felt  inclined  to  make  a  note. 


RESULTS 


Responses  to  each  of  the  questions  are 
discussed  in  this  section.  Appendix  A 
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COMPLETE  REQUEST 


ENTER  AIRCRAFT  IDENTIFICATION 

ENTER  AIRCRAFT  TYPE  &  SPECIAL 
EQUIPMENT  CODE 

ENTER  DESTINATION  IDENTIFIER 

ENTER  ESTIMATED  TIME  EN  ROUTE 

ENTER  AIRCRAFT  TRUE  AIRSPEED 
IN  KNOTS 

ENTER  DEPARTURE  IDENTIFIER 

ENTER  PROPOSED  DEPARTURE  TIME 
IN  LOCAL  OR  GREENWICH 

ENTER  CRUISING  ALTITUDE 

ENTER  ROUTE  OF  FLIGHT 

ENTER  REMARKS 

ENTER  FUEL  ON  BOARD 

ENTER  ALTERNATE  AIRPORT  IDENTIFIER 

ENTER  PILOT'S  NAME,  ADDRESS,  AND 
TELEPHONE  NUMBER,  AND  IDENTIFIER 
FOR  AIRCRAFT  HOME  BASE 

ENTER  NUMBER  OF  PEOPLE  ABOARD 

ENTER  COLOR  OF  AIRCRAFT 


PARTIAL  REQUEST 


BLOCK  1 


FIGURE  6.  OPTIONAL  FLIGHT  PLAN 


BLOCK  2 


BLOCK  3 


BLOCK  4 


BLOCK  5 


BLOCK  6 


BLOCK  7 


BLOCK  8 
BLOCK  9 
BLOCK  10 
BLOCK  11 
■LOCK  12 
BLOCK  13 


BLOCK  14 


BLOCK  15 
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contains  tables  A-l  through  A-8  which 
categorize  the  responses  to  each 
question  according  to  the  following 
subgroups : 

1. '  Engine  rating 

2.  License  rating 

3.  Instrument  rating 

4.  Annual  flying  tine 

5.  Total  flying  time 

RESPONSES  TO  QUESTION  1. 

Question  1  asked,  "Once  faailiar  with 
the  system  and  having  a  flight  plan  form 
in  front  of  you,  what  are  your  feelings 
regarding  the  mode  of  information 
request?"  The  respondent  was  given  two 
choices — the  block  method  of  request 
(partial)  or  the  full  information 
request  (complete) — and  was  asked  to 
explain  the  reason  for  his  choice. 
Table  1  shows  the  distribution  of 
responses  to  this  question.  Using  a 
chi-square  (X2)  test,  no  statistically 
significant  difference  was  found.  A 
large  sample  binomial  sign  test, 
corrected  for  continuity,  confirmed 
this  result.  All  X2  tests  were 
conducted  at  the  0.05  level  of 
significance.  In  using  the  X2  test, 
it  was  assumed  that  if  the  pilots  were 
completely  indifferent,  their  responses 
would  be  tantamount  to  chance. 


The  expected  frequency  on  an  indif¬ 
ference  hypothesis  would  be  25  for 
the  block  method  and  25  for  the  full 
information  method.  This  result  would 
be  expected  by  chance. 

The  respondents  who  selected  the  block 
method  of  request  indicated  that  this 
method  was  preferred  because  it  was 
faster  and  saved  time.  No  other  reason 
was  given  for  this  preference.  On  the 
other  hand,  those  who  selected  the  full 
information  request  offered  a  variety  of 
reasons  for  their  choice.  A  summary 
of  the  reasons  stated  were: 

1.  Does  not  require  the  pilot  to  have 
a  flight  plan  form. 

2.  There  is  less  chance  of  error. 

3.  Communication  is  more  natural. 

4.  Easier  to  understand. 

5.  Does  not  require  the  pilot  to 
remember*  the  flight  plan  format. 

6.  Easier  to  keep  track  of  place  and 
information  to  be  provided. 

7.  Positively  confirms  which  item  is 
being  presented. 


TABLE  1.  RESPONSES  TO  QUESTION  1 


Once  familiar  with  the  system  and  having  a  flight  plan  form  in  front  of 
you,  what  are  your  feelings  regarding  the  mode  of  information  request? 


Block  Method 
Reques  t 
N  Z 


Full  Information 
Request 
N  Z 


29 


58.00 


21 


42.00 


Total 
N  Z 

50  100.00 


1.28 


Z 

-0.99 
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RESPONSES  TO  QUESTION  2. 


Question  2  asked,  "Would  you  use  this 
system  more  if  you  did  not  have  to 
stake  all  entries  by  word  recognition 
but  could  use  the  method  for  recording 
sosm  of  the  entries?"  The  subjects  were 
given  three  choices:  (1)  complete  entry 
by  word  recognition,  (2)  partial  entry 
by  word  recognition  and  partial  entry  by 
recording,  and  (3)  no  preference. 

Table  2  shows  the  distribution  of 
responses  to  this  question.  The 
preponderant  majority  of  pilots  (44  out 
of  50)  preferred  partial  entry  by  word 
recognition  and  partial  entry  by 
recording.  The  X 2  and  Z  values  shown 
in  table  2  were  computed  after  the  no 
preference  responses  were  evenly  split 
between  columns  1  and  2,  These  values 
are  statistically  significant.  We  may, 


therefore,  reject  the  null  hypothesis; 
that  there  was  no  difference  between  the 
entry  methods. 

The  reasons  given  fo^selecting  partial 
entry  by  word  recognition  and  partial 
entry  by  recording  were: 

1.  Requires  less  time  to  file. 

2.  There  is  less  chance  of  error. 

RESPONSES  TO  QUESTION  3. 


Question  3  of  the  schedule  asked,  "With 
repeated  use  do  you  feel  you  could 
adapt  to  filing  a  flight  plan  in  an 
automated  manner?"  The  respondent  was 
given  two  choices — yes  or  no.  Table  3 
shows  the  distribution  of  responses  to 
this  question.  With  the  exception  of 
one  pilot,  all  of  the  subjects  answered 
in  the  affirmative. 


TABLE  2.  RESPONSES  TO  QUESTION  2 


Would  you  use  this  system  more  if  you  did  not  have  to  stake  all  entries  by 
word  recognition  but  could  use  the  method  for  recording  some  of  the  entries? 


Complete  Entry  Partial  Entry  No  Preference  Total 
N  Z  NZ  NZ  NZ 


44  88.00 


3  6.00 


50  100.00 


32.96  5.60 


TABLE  3.  RESPONSES  TO  QUESTION  3 


With  repeated  use  do  you  feel  you  could  adapt  to 
filing  a  flight  plan  in  an  automated  manner? 


Yes 

N  Z 

W  98.00 


No 

N  Z 


Totsl 
N  Z 


1  2,00  50  100.00  46.08 


-6.65 
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RggPOggEg  TO  QUESTION  4. 

Item  4  of  the  questionnaire  stated,  "We 
realize  there  is  a  trade-off  in  using 
any  means  of  filing  a  flight  plan; 
namely,  a  longer  filing  time  in  using 
the  word  recognition  system  for  flight 
plan  filing  versus  possible  long  delays 
in  reaching  the  flight  service  station 
(FSS)  (specialist  or  fast  file)  to  file 
by  phone.  Delay  time  in  reaching  the 
FSS  by  phone  will  vary  depending  on 
current  weather  in  the  FSS  service 
area  and  geographic  location  of  the 
FSS  (very  long  phone  waits  in  the 
New  York  City  area  during  periods  of 
marginal  weather).  Considering  all  the 
above  factors,  which  would  be  your 
preferred  method  of  filing  future  flight 
plans?" 

The  respondent  was  given  two  choices, 
the  longer  time  using  the  word  recogni¬ 
tion  system  or  a  phone  call  directly  to 
the  specialist.  As  shown  in  table  4, 
29  pilots  selected  the  longer  time  using 
word  recognition,  20  selected  a  phone 
call  directly  to  the  specialist,  and  1 
did  not  select  either  of  the  fixed 


TABLE  4.  RESPONSES  TO  QUESTION  4 

We  realize  there  is  a  trade-off  in  using  any  means  of  filing  flight  plans;  namely, 
a  longer  filing  time  in  using  the  word  recognition  system  for  flight  plan  filing 
versus  possible  long  delays  in  reaching  the  FSS  (specialist  or  fast  file)  to  file 
by  phone.  Delay  time  in  reaching  the  FSS  by  phone  will  vary  depending  on  current 
weather  in  the  FSS  service  area  and  geographic  location  of  the  FSS  (very  long  phone 
waits  in  the  NYC  area  during  periods  of  marginal  weather).  Considering  all  the 
above  factors,  which  would  be  your  preferred  method  of  filing  future  flight  plans? 

Word  Recognition 

System  Phone  Call  Total  _ 

NX  N  Z  N  Z  )T  Z 


29  59.18  20  40.82  49*  100.00  1.65  -1.14 


choices.  The  X*  iB  equal  to  1.65 
which  is  not  statistically  significant 
at  the  0.05  level.  A  large  sample 
binomial  sign  test  confirmed  this 
result — Z  ■  -1.14.  A  number  of  the 
pilots  who  indicated*  a  preference  for 
filing  flight  plans  by  computerized 
word  recognition  qualified  their 
responses  by  adding,  "If  specialist  not 
readily  available .. .If  the  phone  is 
busy  to  the  specialist ..  .If  it  allows 
me  to  get  through  quicker...,"  etc. 

Many  pilots  felt  that  question  4 
required  more  thought  than  the  others 
because  of  the  requirement  to  choose 
between  the  URD  and  the  FSS  specialist 
for  filing.  The  difficulty  arose 
because  under  the  present  systen, 
weather  and  other  aeronautical  data  are 
normally  procured  at  the  same  time  the 
flight  plan  is  filed.  It  was  explained 
to  the  pilots  that  this  test  was  only 
concerned  with  the  flight  plan  filing 
function  and  not  with  weather  or  Notice 
to  Airmen  data.  However,  they  had 
trouble  divorcing  these  functions.  They 
questioned  what  would  be  gained  timewise 
by  filing  through  the  URD  when  a  second 


*0ne  respondent  did  not  select  either  of  the  fixed  choices.  Instead  he  stated, 
"answer  depends  on  which  way  is  longer."  He  indicated  that  the  word  recognition 
ayatem  was  "preferred  if  overall  time  saving  is  significant,"  and  a  phone  call 
direct  to  specialist  was  preferred  if  time  difference  is  tolerable  or  not 
significant. 


call  would  still  have  to  be  wade  to  the 
special iat  which  could  require  a  further 
wait.  The  pilots  who  expressed  this 
concern  suggested  that  some  type  of 
selected  weather  data  be  included  as  an 
UKD  option.  Offering  limited  weather 
information  as  an  URD  option  could 
significantly  reduce  the  time  spent  with 
the  specialist  and  also  reduce  the 
nuad>er  of  amended  flight  plans,  since 
the  pilot  could  make  better  initial 
decisions  on  altitude  or  other  elements. 

RESPONSES  TO  QPESTION  5. 

Question  5  asked  the  pilot,  "On  the 
manner  of  prompt,  do  you  prefer  the 
cue-tone  prompt,  the  word  echo-back 
prompt,  or  no  prompt?"  Of  the  fifty 
subjects,  not  one  selected  the  cue-tone 
prompt.  The  majority  of  respondents 
selected  the  no  prompt  choice.  Table  5 
shows  the  distribution  of  responses. 
Since  the  X2  and  Z  values  were 
stat iat ical 1/  significant,  the  null- 
hypothesis,  that  there  was  no  difference 
between  methods  of  prompting,  may  be 
rejected. 


The  beep  tone  used  in  flight  plan  A 
(figure  3)  w*e  evidently  quite 
irritating  to  the  users.  When  all  three 
tests  were  completed,  they  were  asked  to 
tentatively  select  the  preferred  method 
while  all  were  f resh . in  their  minds. 
The  cue-tone  prompt  (method  A)  was 
immediately  rejected  without  hesitation 
by  most  subjects.  The  sound  of  the  tone 
was  undesirable. 

The  outstanding  feature  of  the  word 
echo-back  prompt  (method  B)  was  the 
ability  to  hear  immediate  confirmation 
of  the  word  entered  and  the  ability  to 
immediately  enter  a  correction.  No 
prompt  (method  C,  preferred)  had  the 
desirable  effect  of  allowing  a  more 
natural  conversational  style  of  entry 
which  had  a  very  strong  appeal  with  most 
subjects. 

The  use  of  the  command  word  LAST  to 
correct  previous  lines  was  considered 
useful  and  was  used  quite  often  during 
testing.  Many  pilots  used  it;  however, 
several  admitted  employing  it 
unnecessarily  just  to  see  how  it  would 
work. 


TABLE  5.  RESPONSES  TO  QUESTION  5 


On  the  manner  of  prompt,  do  you  prefer  the  cue-tone 
prompt,  the  word  echo-back  prompt,  or  no  prompt? 


Cue-Tone 

Word  Echo-Back 

Prompt 

Prompt 

No  Prompt 

Total 

o 

N  X 

N 

X 

N 

X 

N  X 

X 

Z 

0  0 

16 

32.00 

34 

68.00 

SO  100.00 

6.48 

2.40 
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RESPONSES  TO  QUESTION  6. 

Item  6  of  the  questionnaire  asked, 
"Is  the  available  vocabulary  sufficient 
or  would  more  command  words  better 
enable  flight  plan  entry?"  The 
respondents  were  given  three  choices  as 
follows: 

1.  Present  vocabulary  sufficient. 

2.  Need  changes  in  present  vocabulary 
as  noted. 

3.  Present  vocabulary  insufficient, 
should  add. 

The  X*  and  Z  values  shown  in  table  6 
indicate  a  test  of  the  significance 
of  the  difference  between  the  number  of 
responses  in  column  1  (Vocabulary 
Sufficient)  and  the  total  number  of 
responses  in  column  2  (Need  Changes)  and 
column  3  (Vocabulary  Insufficient).  No 
statistically  significant  difference  was 
found. 

The  following  changes  and  additions  to 
the  vocabulary  were  suggested: 

1.  Use  slash  instead  of  slant. 


A  significant  problem  encountered  during 
testing  involved  nonstandard  words  in 
the  phonetic  alphabet.  Many  of  the 
older,  more  experienced  pilots  were 
using  the  old  phonetic  alphabet,  which 
generated  a  lot  of  mistakes.  In  fact, 
before  testing  beg^n  many  pilots 
confessed  to  being  a  "little  rusty,"  or 
some  similar  statement,  regarding  use 
of  the  phonetic  alphabet.  This 
opinion  was  expressed  by  pilots  in  all 
categories  of  experience,  but  the  older 
group  predominated.  Several  pilots 
suggested  programing  several  of  the 
"old"  phonetic  words  along  with  the 
correct  ones.  This  might  consist  of 
half  a  dozen  or  so  of  the  words  most 
likely  to  present  a  problem  because  of 
their  widespread,  though  incorrect,  use. 
To  avoid  confusion,  these  nonstandard 
words  would  not  be  advertised  as 
existing  in  the  system,  but  would 
nevertheless  be  programed  and  available 
anytime  a  pilot  lapsed  into  old  ways.  A 
significant  proportion  of  errors 
might  be  avoided  this  way.  A  comparison 
of  the  phonetically  correct  word  with 
examples  of  what  many  pilots  tended  to 
use  is  shown  below.  Each  example  was 
used  at  least  once,  and  the  asterisk 
indicates  frequent  usage. 


2.  Add  the  command  word  "thousand"  for  PHONETIC  ALPHABET  COMPARISON 

altitudes  and  "hundred"  for  time  and 

fuel  instead  of  successive  zeros.  Correct  Often  Used 


3.  Add  common  aircraft  manufacturers  Alpha . . . Able 

names,  such  as  Cessna,  Piper,  etc.  *  Bravo . Baker 

Delta . Dog 

4.  Add  the  words  "knots,"  "zulu,"  *  Echo . Easy 

"hours,"  "minutes,"  "single  engine,"  Golf . George 

and  "over  water."  *  India . Item 

Juliett . . . Jig 

5.  Fuel  on  board  expressed  as  "four  Kilo . King 

four  five"  is,  not  as  meaningful  as  November . Nan 

4  hours  and  45  minutes.  *  Sierra . Sugar 

Uniform . .Uncle 

6.  Add  multiple  variations  of  the  Yankee . Yoke 

phonetic  alphabet.  Zulu . . . Zebra,  Zed 

7.  Use  the  command  word  "none"  or  In  a  few  cases,  there  were  problems 

"blank"  to  signify  no  entry.  with  dialects,  accents,  or  personnel 

id  iosyncrac ies .  For  example,  one 

Table  6  shows  the  distribution  of  individual  used  Limo  for  Lima,  and 

responses  to  question  6.  Sierro  for  Sierra. 
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TABLE  6. 


RESPONSES  TO  QUESTION  6 


Is  the  available  vocabulary  sufficient  or  would  more 
command  words  better  enable  flight  plan  entry? 


Vocabulary 

Sufficient 

N  X 

Need 

Changes 

N  X 

Vocabulary 
Insuff ic ient 

N  X 

Total 

N  X 

Z 

30  58.82 

6  11.76 

15  29.41 

51*  100.00 

1.59 

-1.12 

*One  respondent  selected  two  of  the  three  alternatives— "Need  Changes" 
and  "Vocabulary  Insufficient." 


RESPONSES  TO  QUESTION  7. 

Question  7  asked  the  pilot,  "Do  you  feel 
that  you  could  successfully  file  a 
flight  plan  using  this  word  recognition 
system?"  The  subjects  were  given  a 
choice  of  a  "yes"  or  "no"  in  responding 
to  this  question.  As  shown  in  table  7, 
the  overwhelming  majority  of  pilots 
(49  out  of  50)  answered  "yes"  to  this 
question.  However,  2  of  the  49  pilots 
qualified  their  answer  as  follows: 


pilots)  answered  "yes"  to  question  8. 
It  should  be  noted,  however,  that  7  out 
of  the  46  pilots  qualified  their  answer 
with  the  following  statements: 

1.  "Yes ,  if  this  was  all  that  was 
available." 

2.  "Only  if  no  other  choice." 

3.  "Yes,  if  the  alternative  meant 
excessive  delay." 


1.  "With  more  practice." 

2.  "It  can  be  done,  I  would  not  like 
this  type  of  filing  a. flight  plan." 

RESPONSES  TO  QUESTION  8. 

Item  8  of  the  questionnaire  asked  the 
pilot,  "Would  you  use  a  filing  system 
similar  to  this?"  The  answer  to  this 
question  was  limited  to  a  "yes"  or  "no" 
response.  Table  8  shows  that  the 
overwhelming  majority  (46  out  of  50 


4.  "Yes,  if  specialist  not  available." 

5.  "Yes— probably. " 

6.  "Depending  upon  location." 

7.  "Yes,  if  I  had  trouble  getting  a  FSS 
person." 

There  were  no  qualifying  answers  from 
any  of  the  4  pilots  who  answered  "no"  to 
question  8. 
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TABLE  7.  RESPONSES  TO  QUESTION  7 


Do  you  feel  Chat  you  could  successfully  file  a 
flight  plan  using  this  word  recognition  system?  . 


Yes 

N 


X 


Total 

N  X  \ 


49  98.00 


1  2.00  50  100.00  46.08 


6.65 


TABLE  8.  RESPONSES  TO  QUESTION  8 


Would  you  use  a  filing  system  similar  to  this? 


Yes 

N  X 


46 


92.00 


No 


_N_ 

4 


X 

8.00 


Total 
N  X 


50 


100.00 


35.28 


Z 

-5.80 


RESPONSES  TO  ITEM  9. 

Item  9  of  the  questionnaire  invited  the 
respondents  to  write  in  any  comments 
they  wished  to  make.  The  comments 
received  provide  important  insights  into 
th  attitudes  of  the  subject  pilots 
Coward  automatic  flight  plan  filing. 
For  this  reason  they  are  provided  in 
this  report  as  actually  stated  rather 
than  summarized.  Except  for  some 
comments  that  were  eliminated  because 
they  were  not  considered  to  be  pertinent 
or  informative,  the  following  is  a  list 
of  the  write-in  comments  received. 


1.  "Reference  item  4,  the  longer  times 
using  word  recognition.  Typically,  there 
are  'delays  in  obtaining  the  specialist 
to  the  extent  that  word  recognition 
would  imply  a  longer  period  on  the  phone 
but  less  problems  completing  the 
phone  call.  Also,  on  occasions  where 
weather  information  has  been  obtained; 
otherwise,  there  is  no  need  to  call  the 
specialist ." 

2.  "I  felt  better  with  the  echo  prompt 
than  with  the  others — although  I  felt 
the  no  prompt  would  be  faster." 
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3.  "I  preferred  no  prompter  system 
because  the  repeated  information  was 
longer  (continuity)  as  opposed  to  short 
segments  of  repeated  information.  More 
concentration  and  time  appeared  to 
be  required  for  flight  plans  1  and  2 
(especially  No.  1)." 

4.  "A  specialist  would  advise  one  on 
new  (weather)  data.  A  computer  would 
not.  I  feel  the  computer  has  a  future 
in  flight  planning.  However,  it  could 
not  replace  the  TRUST  1  receive  from  a 
specialist.  I  like  to  see  weather 
observation  tied  in  with  computerized 
flight  planning." 

5.  "If  no  form  available,  prefer  full 
info  request." 

6.  "Recent  flight  experience  with  USDA 
screwworm  eradication  in  Mexico.  The 
area  we  flew  in  has  no  NAVAIDS  to  speak 
of  and  our  only  radio  contact  was  with 
FSS  and  we  were  totally  dependent  on  FSS 
for  flight  following.  Replacement  of 
the  FSS  specialist  would  affect  safety 
of  this  particular  program." 


these  aggravated  delays  at  flight  plan 
filing  telephones." 

12.  "Sometimes  excessive  repeating  to 
voice  computer  of  its  mistakes  could  be 
most  disturbing  mentally.  Could  even 
cause  pilots  in  disgust  to  cancel  1FR 
flight  plans  and  jeopardize  flight 
safety." 

13.  "Time  is  important,  any  means  to 
reduce  time  without  compromising 
accuracy  should  be  tried." 

14.  "Regarding  No.  4:  I  feel  I  would 
prefer  to  file  a  flight  plan  directly 
with  a  specialist  at  FSS.  It  just  seems 
more  personalized  talking  with  another 
person.  Perhaps  a  combination  could 
be  worked  out  whereby  the  computer 
filing  would  be  used  during  periods  of 
high  workloads  on  the  specialists  and 
directly  with  people  when  they  are 
available." 

15.  "Liked  flight  plan  three  the  best. 
I  could  get  used  to  it  and  take  very 
little  longer  than  method  now  used." 


7.  "Once  the  flight  plan  is  in  proper 
format  in  front  of  you  the  automation  of 
reading  into  phone  should  be  no  problem. 
In  fact,  I  prefer  this  method." 

8.  "Silence  as  a  terminator  makes  a 
demand  not  easily  met  at  some  phone 
stations  (noisy  offices,  outside  phone 
booths,  etc.)." 

9.  "We  need  to  guard  against  adding 
user  burdens  in  an  attempt  to  ease 
the  burdens  of  the  service  elements." 

10.  "Presently  FSS  phones  at  FSS's  are 
limited  and  line  ups  exist  at  high 
density  airports,  time  delay  in  getting 
through  to  computer — if  as  bad  as 
getting  through  to  SPECIALIST  would  be 
prohibitive." 

11.  "I  can  foresee  tempers  flaring,  if 
executive  pilots  have  to  put  up  with 


16.  "The  read  back  of  departure  time, 
fuel  aboard,  and  routing,  plus  alternate 
airport  would  save  time  on  flight  plan." 

17.  "Background  noise  could  be  a 
problem  for  filing  in  some  locations." 

18.  "Somewhere  downstream  why  can't  we, 
jus  t  for  example,  just  give  departure 
and  destination  airports  and  just 
request  appropriate  routes." 

19.  "Biggest  problem  was  corrections 
for  me,  and  where  to  jump  in  for 
correction — I  would  always  like  to  hear 
play  back  corrections." 

20.  "Has  the  thought  of  using  CRT 
display  of  typed  entry  been  explored?" 

21.  "When  computer  asks  to  please 
repeat  or  correction  is  made,  have 
computer  read  back  whole  segment." 


22.  "I  think  the  coaputer  would  need  to 
be  proven  reliable." 

23.  "Much  in£o  could  be  'canned'  for 
each  aircraft,  thereby  shortening 
flight  plan." 

Important  iteaa: 

Destination 
Aircraft  type 
Departure  point 

24.  "I  aa  an  Aray  Guard  pilot.  We  fill 
out  a  175-1  weather  briefing  before  the 
175  (flight  plan).  This  would  mean  that 
a  coaputer  giving  weather  info  would 
have  to  be  separate  from  one  accepting 
flight  plans." 

25.  "Coaputer  had  aoae  difficulty 
understanding  ee  on  flight  plan  No.  2. 
Proper  enunciation  is  important  and 
pilots  (foreign  accent)  would  have  to  be 
briefed.  Flight  plan  No.  2 — On  two 
entries,  altitude  9,000  feet  and  /BWZ/, 
I  was  not  sure  whether  the  coaputer 
properly  accepted  9,000  feet  and  W." 

26.  "If  you  can  speed  up  filing  flight 
plans  in  areas  like  DCA  and  BAL,  I  am 
all  for  it." 

27.  "I  use  the  system  (FSS)  almost 
daily  and  have  experienced  little  or 
no  delays  in  getting  my  flight  plan 
filed." 

28.  "I  want  the  luxury  of  discussing 
the  WX  with  the  flight  service 
specialists." 

29.  "Would  like  to  see  the  Pireps  for 
my  route  of  flight  included  at  the 
end  of  flight  plan." 

30.  "The  system  at  present  to  me  is 
fine — it  keeps  a  personal  touch  and 
it  feels  better.  Also,  if  there  is  a 
aistake,  you  can  talk  to  a  person." 

31.  "Reference  No.  4 — For  special 
missions  such  as  our  flight  testing  at 


NAFBC,  this  automated  system  is  not 
usable.  For  example,  we  may  have  to  fly 
unusual  patterns  both  vertically  and 
horizontally  at  varying  speeds,  etc." 

32.  "Need  specialist  for  pilot  reports 
and  specific  weather  en  route  briefing, 
rather  than  getting  a  weather  report  for 
large  section  of  country." 


CONCLUSIONS 


Given  the  assumption  that  the  actual 
utterance  recognition  device  performs  at 
a  level  comparable  to  that  of  the 
simulation  used  in  this  experiment,  it 
is  reasonable  to  conclude  that: 

1.  General  aviation  pilots  could,  and 
would,  elect  to  file  fl  ight  plans 
by  computerized  word  recognition  using  a 
real  computer. 

2.  General  aviation  pilots  would 
prefer  partial  entry  by  word  recognition 
and  partial  entry  by  recording  over 
complete  entry  of  information  via  word 
recognition  when  speaking  to  a  real 
Computer. 

3.  General  aviation  pilots  would  prefer 
the  no  prompt  method  or  word  echo-back 
prompt  over  the  cue-tone  prompt  when 
speaking  to  a  real-word  recognition 
computer. 

4.  General  aviation  pilots  would 
be  able  to  file  flight  plans  by 
computerized  word  recognition  using  the 
experimental  vocabulary  when  speaking 
to  a  real  computer.  We  cannot  offer 
any  compelling  evidence  at  this  time 
regarding  the  optimum  size  and  content 
of  the  word  recognition  vocabulary. 
Further  testing  is  necessary  to 
ascertain  this  information. 

Our  results  show  a  small  absolute 
difference  between  the  preference  for 
the  block  method  of  request  and  the  full 
information  request;  with  the  lsrger 
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number  of  subjects  preferring  the 
block  method.  The  difference  is  not 
statistically  reliable.  Therefore,  we 
cannot  offer  any  compelling  evidence  at 
this  time  that  there  is  any  profound 
difference  in  the  preference  for  either 
method. 

Our  results  show  a  small  difference 
between  the  preference  for  using  the 
word  recognition  system  over  calling 
the  specialist  directly  (given  the 
possible  long  delays  in  reaching  the 
Flight  Service  Station  (FSS).  The 
difference,  however,  is  not  statisti¬ 
cally  reliable.  Therefore,  we  cannot 
offer  any  compelling  evidence  at 
this  time  that  there  is  any  profound 
difference  in  pilot  preference. 

RECOMMENDATIONS 


1.  It  is  recommended  that  additional 
experiments  be  conducted  using  a 
real  utterance  recognition  device  (URD) 
to  determine: 


a.  The  performance  degradation  due 
to  an  expanded  vocabulary. 

b.  The  effects  of  regional 
differences  in  pronunciations. 

c.  The  optimum  size  and  content  of 
the  vocabulary. 

d.  The  best  protocol  for  entering 
the  data. 

e.  The  attitude  of  the  general 
aviation  public  toward  this  system. 

2.  Once  the  limitations  of  the 
utterance  recognition  device  are  fully 
known,  an  adaptation  strategy  should  be 
developed  to  overcome  these  limitations. 

3.  In  future  evaluation  experiments, 
make  provision  for  a  limited  amount 
of  weather  or  Notice  to  Airmen  (NOTAM) 
data  to  be  available  to  the  pilot  as 
an  URD  option.  (See  commentary  on 
question  4. ) 


TABLE  A-l 


RESPONSES  TO  QUESTION  1  ACCORDING  TO  SUBGROUP 


Block  Method 

Full 

Information 

Request 

N  Z 

N 

Request 

Z 

N 

Total 

Z 

“  — — — 

‘  ■ 

Engine  Rating 

Single  Engine 

10 

62.50 

6 

37.50 

16 

100.00 

Multi  Engine 

19 

55.88 

15 

44.12 

34 

100.00 

Total 

29 

58.00 

21 

42.00 

50 

100.00 

License  Rating 

Private 

3 

42.86 

4 

57.14 

7 

100.00 

Commercial 

14 

60.87 

9 

39.13 

23 

100.00 

ATP 

12 

60.00 

8 

40.00 

20 

100.00 

Total 

29 

58.00 

21 

42.00 

50 

100.00 

Instrument  Rating 

IFR 

22 

57.89 

16 

42.11 

38 

100.00 

VFR 

7 

58.33 

5 

41.67 

12 

100.00 

Total 

29 

58.00 

21 

42.00 

50 

100.00 

Annual  Flying  Time 

0-100  hours 

14 

50.00 

14 

50.00 

28 

100.00 

>  100  hours 

15 

68.18 

7 

31.82 

22 

100.00 

Total 

29 

58.00 

21 

42.00 

50 

100.00 

Total  Flying  Time 

1-999 

7 

50.00 

7 

50.00 

14 

100.00 

1,000-3,999 

9 

64.29 

5 

35.71 

14 

100.00 

4,000-10,000 

7 

70.00 

3 

30.00 

10 

100.00 

>  10,000 

6 

50.00 

6 

50.00 

12 

100.00 

Total 

29 

58.00 

21 

42.00 

50 

100.00 

TABLE  A-2.  RESPONSES  TO  QUESTION  2  ACCORDING  TO  SUBGROUP 


Complete  Entry  Partial  Entry  No  Preference  Total 

NX  NX  NXNXZ* 


Engine  Rating 


Single  Engine 

1 

5.88 

15 

88.24 

1 

5.88 

17 

100.00 

3.06 

Multi  Engine 

2 

6.06 

29 

87.88 

2 

6.06 

33 

100.00 

4.53 

Total 

3 

6.00 

44 

88.00 

3 

6.00 

50 

100.00 

5.60 

License  Rating 

Private 

0 

0 

7 

100.00 

0 

0 

7 

100.00 

2.27 

Commercial 

1 

4.35 

20 

86.96 

2 

8.70 

23 

100.00 

3.75 

ATP 

2 

10.00 

17 

85.00 

1 

5.00 

20 

100.00 

2.92 

Total 

3 

6.00 

44 

88.00 

3 

6.00 

50 

100.00 

5.60 

Instrument  Rating 

IFR 

2 

5.26 

34 

89.47 

2 

5.26 

38 

100.00 

5.03 

VFR 

1 

8.33 

10 

83.33 

1 

8.33 

12 

100.00 

2.41 

Total 

3 

6.00 

44 

88.00 

3 

6.00 

50 

100.00 

5.60 

Annual  Flying  Time 

0-100 

1 

3.57 

25 

89.29 

2 

7.14 

28 

100.00 

4.35 

>  100 

2 

9.09 

19 

86.36 

1 

4.55 

22 

100.00 

3.20 

Total 

3 

6.00 

44 

88.00 

3 

6.00 

50 

100.00 

5.60 

Total  Flying  Time 

0-999 

1 

7.14 

13 

92.86 

0 

0 

14 

100.00 

2.94 

1,000-3,999 

0 

0 

12 

85.71 

2 

14.29 

14 

100.00 

2.94 

4,000-10,000 

1 

10.00 

9 

90.00 

0 

0 

10 

100.00 

2.21 

>  10,000 

1 

8.33 

10 

83.33 

1 

8.33 

12 

100.00 

2.41 

Total 

3 

6.00 

44 

88.00 

3 

6.00 

50 

100.00 

5.60 

♦All  Z  valuea  significant  at  the  0.05  level 


TABLE  A- 3 


RESPONSES  TO  QUESTION  3  ACCORDING  TO  SUBGROUP 


Yes 

No 

Total 

N 

% 

N 

X 

N 

X 

z* 

" 

' 

—  — ' 

' 

— 

Engine  Rating 

Single  Engine 

16 

100.00 

0 

0 

16 

100.00 

-3.75 

Multi  Engine 

33 

97.06 

1 

2.94 

34 

100.00 

-5.32 

Total 

49 

98.00 

1 

2.00 

50 

100.00 

-6.65 

License  Rating 

Private 

7 

100.00 

0 

0 

7 

100.00 

-2.27 

Commercial 

23 

100.00 

0 

0 

23 

100.00 

-4.59 

ATP 

19 

95.00 

1 

5.00 

20 

100.00 

-3.80 

Total 

49 

98.00 

1 

2.00 

50 

100.00 

-6.65 

Instrument  Rating 

IFR 

37 

97.37 

1 

2.63 

38 

100.00 

-5.68 

VFR 

12 

100.00 

0 

0 

12 

100.00 

-3.18 

Total 

49 

98.00 

1 

2.00 

50 

100.00 

-6.65 

Annual  Flying  Time 

0-100  hours 

28 

100.00 

0 

0 

28 

100.00 

-5.10 

>  100  hours 

21 

95.45 

1 

4.55 

22 

100.00 

-4.05 

Total 

49 

98.00 

1 

2.00 

50 

100.00 

-6.65 

Total  Flying  Time 

0-999 

14 

100.00 

0 

0 

14 

100.00 

-3.47 

1,000-3,999 

14 

100.00 

0 

0 

14 

100.00 

-3.47 

4,000-10,000 

10 

100.00 

0 

0 

10 

100.00 

-2.85 

>  10,000 

11 

91.67 

1 

8.33 

12 

100.00 

-2.60 

Total 

49 

98.00 

1 

2.00 

50 

100.00 

-6.65 

*All  Z  values  are  significant  at  the  0.05  level 
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TABLE  A-4.  RESPONSES  TO  QUESTION  4  ACCORDING  TO  SUBGROUP 


Word  Recognition 

Phone 

Call 

Total 

N 

Z 

N 

Z 

N 

Z 

1-1 

""  ■■ 

— 

1 

Engine  Rating 

Single  Engine 

11 

68.75 

5 

31.25 

16 

100.00 

Multi  Engine 

18 

54.55 

15 

45.45 

33 

100.00 

Total 

29 

59.18 

20 

40.82 

49 

100.00 

License  Rating 

Private 

4 

57.14 

3 

42.86 

7 

100.00 

Commercial 

17 

73.91 

6 

26.09 

23 

100.00 

ATP 

8 

42.11 

11 

57.89 

19 

100.00 

Total 

29 

59.18 

20 

40.82 

49 

100.00 

Instrument  Rating 

IFR 

22 

59.46 

15 

40.54 

37 

100.00 

VFR 

7 

58.33 

5 

41.67 

12 

100.00 

Total . 

29 

59.18 

20 

40.82 

49 

100.00 

Annual  Flying  Time 

0-100  hours 

21 

75.00 

7 

25.00 

28 

100.00 

>100  hours 

8 

38.10 

13 

61.90 

21 

100.00 

Total 

29 

59.18 

20 

40.82 

49 

100.00 

Total  Flying  Time 

0-999 

9 

64.29 

5 

35.71 

14 

100.00 

1,000-3,999 

11 

84.62 

2 

15.38 

13 

100.00 

4,000-10,000 

5 

45.45 

6 

54.55 

11 

100.00 

>  10,000 

4 

36.36 

7 

63.64 

11 

100.00 

Total  , 

29 

59.18 

20 

40.82 

49 

100.00 

*Significant  at  tha  0.05  level. 


A-4 


Z 


-1.25 

-0.35 

-1.14 


0 

-2.09* 

0.46 

-1.14 


-0.99 

-0.29 

-1.14 


-2.46* 

0.87 

-1.14 


-0.80 

-2.22* 

0 

0.60 

-1.14 


TABLE  A- 5 


RESPONSES  TO  QUESTION  5  ACCORDING  TO  SUBGROUP 


Cue-Tone 

Prompc 

Word  Echo-Back 
Prompt 

No 

Prompt 

Total 

N 

X 

N 

X 

N 

X 

N 

X 

' 

" 

Engine  Rating 

Single  Engine 

0 

0 

5 

31.25 

11 

68.75 

16 

100.00 

Multi  Engine 

0 

0 

11 

32.35 

23 

67.65 

34 

100.00 

Total 

0 

0 

16 

32.00 

34 

68.00 

50 

100.00 

License  Rating 

Private 

0 

0 

3 

42.86 

4 

57.14 

7 

100.00 

Commercial 

0 

0 

7 

30.43 

16 

69.57 

23 

100.00 

ATP 

0 

0 

6 

30.00 

14 

70.00 

20 

100.00 

Total 

0 

0 

16 

32.00 

34 

68.00 

50 

100.00 

Instrument  Rating 

IFR 

0 

0 

12 

31.58 

26 

68.42 

38 

100.00 

VFR 

0 

0 

4 

33.33 

8 

66.67 

12 

100.00 

Total 

0 

0 

16 

32.00 

34 

68.00 

50 

100.00 

Annual  Flying  Time 

0-100 

0 

0 

11 

39.29 

17 

60.71 

28 

100.00 

>  100 

0 

0 

5 

22.73 

17 

77.27 

22 

100.00 

Total 

0 

0 

16 

32.00 

34 

68.00 

50 

100.00 

Total  Flying  Time 

0-999 

0 

0 

6 

42.86 

8 

57.14 

14 

100.00 

1,000-3,999 

0 

0 

4 

28.57 

10 

71.43 

14 

100.00 

4, 000-10,0/10 

0 

0 

2 

20.00 

8 

80.00 

10 

100.00 

>  10,000 

0 

0 

4 

33.33 

8 

66.67 

12 

100.00 

Total 

0 

0 

16 

32.00 

34 

68.00 

50 

100.00 

Z 


1.25 

1.89 

2.40* 


0 

1.67 

1.57 

2.40* 


2.11* 

0.87 

2.40* 


0.94 

2.35* 

2.40* 


0.27 

1.34 

1.58 

0.87 

2.40* 


*Significant  at  tha  0.05  leva! 


TABLE  A-6 .  RESPONSES  TO  QUESTION  6  ACCORDING  TO  SUBGROUP 


Vocabulary 
Suffic ient 

N  X 

Need 

Changes 

N  X 

Vocabulary 

Insufficient 

N  X 

Total 

N  X 

z* 

1 

“  * 

" 

' 

“  “  “ 

—  — 

Engine  Rating 

Single  Engine 

11 

68.75 

1 

6.25 

4 

25.00 

16 

100.00 

-1.25 

Multi  Engine 

19 

54.29 

5 

14.29 

11 

31.43 

35 

100.00 

-0.34 

Total 

30 

58.82 

6 

11.76 

15 

29.41 

51 

100.00 

-1.12 

License  Rating 

Private 

4 

57.14 

1 

14.29 

2 

28.57 

7 

100.00 

0 

Commercial 

15 

60.00 

2 

8.00 

8 

32.00 

25 

100.00 

-0.80 

ATP 

11 

57.89 

3 

15.79 

5 

26.32 

19 

100.00 

-0.46 

Total 

30 

58.82 

6 

11.76 

15 

29.41 

51 

100.00 

-1.12 

Instrument  Rating 

IFR 

22 

57.89 

4 

10.53 

12 

31.58 

38 

100.00 

-0.81 

VFR 

8 

61.54 

2 

15.38 

3 

23.08 

13 

100.00 

-0.55 

Total 

30 

58.82 

6 

11.76 

15 

29.41 

51 

100.00 

-1.12 

Annual  Flying  Time 

0-100 

18 

62.07 

2 

6.90 

9 

31.03 

29 

100.00 

-1.11 

>  100 

12 

54.55 

4 

18.18 

6 

27.27 

22 

100.00 

-0.21 

Total 

30 

58.82 

6 

11.76 

15 

29.41 

51 

100.00 

-1.12 

Total  Flying  Time 

0-999 

7 

50.00 

1 

7.14 

6 

42.86 

14 

100.00 

0 

1,000-3,999 

9 

69.23 

1 

7.69 

3 

23.08 

13 

100.00 

-1.11 

4,000-10,000 

8 

66.67 

3 

25.00 

1 

8.33 

12 

100.00 

-0.87 

>  10,000 

6 

50.00 

1 

8.33 

5 

41.67 

12 

100.00 

0 

Total 

30 

58.82 

6 

11.76 

15 

29.41 

51 

100.00 

-1.12 

*Z  scores  indicate  a  test  of  the  significance  of  the  difference  between  the  number 
of  reaponaes  in  column  1  (Vocabulary  Sufficient)  and  the  total  number  of  responses 
in  columns  2  and  3  (Need  Changes)  and  (Vocabulary  Insufficient). 


TABLE  A- 7 


RESPONSES  TO  QUESTION  7  ACCORDING  TO  SUBGROUP 


N 

Yea 

X 

N 

No 

X 

Total 

N  X 

z* 

" 

— 

Engine  Rating 

Single  Engine 

16 

100.00 

0 

0 

16 

100.00 

-3.75 

Multi  Engine 

33 

97.06 

1 

2.94 

34 

100.00 

-5.32 

Total 

49 

98.00 

1 

2.00 

50 

100.00 

-6.65 

License  Rating 

Private 

7 

100.00 

0 

0 

7 

100.00 

-2.27 

Commercial 

23 

100.00 

0 

0 

23 

100.00 

-4.59 

ATP 

19 

95.00 

1 

5.00 

20 

100.00 

-3.80 

Total 

49 

98.00 

1 

2.00 

50 

100.00 

-6.65 

Instrument  Rating 

IFR 

37 

97.37 

1 

2.63 

38 

100.00 

-5.68 

VFR 

12 

100.00 

0 

0 

12 

100.00 

-3.18 

Total 

49 

98.00 

1 

2.00 

50 

100.00 

-6.65 

Annual  Flying  Time 

0-100 

28 

100.00 

0 

0 

28 

100.00 

-5.10 

>  100 

21 

95.45 

1 

4.55 

22 

100.00 

-4.05 

Total 

49 

98.00 

1 

2.00 

50 

100.00 

-6.65 

Total  Flying  Time 

0-999 

14 

100.00 

0 

0 

14 

100.00 

-3.47 

1,000-3,999 

13 

92.86 

1 

7.14 

14 

100.00 

-2.94 

4,000-10,000 

10 

100.00 

0 

0 

10 

100.00 

-2.85 

>  10,000 

12 

100.00 

0 

0 

12 

100.00 

-3.18 

Total 

49 

98.00 

1 

2.00 

50 

100.00 

-6.65 

♦All  Z  values  significant  at  the  0.05  level. 


TABLE  A-8.  RESPONSES  TO  QUESTION  8  ACCORDING  TO  SUBGROUP 


Yea 

No 

Total 

N 

X 

N 

X 

N 

X 

z* 

" 

' 

' 

“  1-1  1  " 

" 

Engine  Rating 

Single  Engine 

16 

100.00 

0 

0 

16 

100.00 

-3.75 

Multi  Engine 

30 

88.24 

4 

11.76 

34 

100.00 

-4.29 

Total 

46 

92.00 

4 

8.00 

50 

100.00 

-5.80 

License  Rating 

Private 

7 

100.00 

0 

0 

7 

100.00 

-2.27 

Commercial 

23 

100.00 

0 

0 

23 

100.00 

-4.59 

ATP 

16 

80.00 

4 

20.00 

20 

100.00 

-2.46 

Total 

46 

92.00 

4 

8.00 

50 

100.00 

-5.80 

Instrument  Rating 

IFR 

34 

89.47 

4 

10.53 

38 

100.00 

-4.70 

VFR 

12 

100.00 

0 

0 

12 

100.00 

-3.18 

Total 

46 

92.00 

4 

8.00 

50 

100.00 

-5.80 

Annual  Flying  Time 

0-100 

28 

100.00 

0 

0 

28 

100.00 

-5.10 

>  100 

18 

81.82 

4 

18.18 

22 

100.00 

-2.77 

Total 

46 

92.00 

4 

8.00 

50 

100.00 

-5.80 

Total  Flying  Time 

0-999 

14 

100.00 

0 

0 

14 

100.00 

-3.47 

1,000-3,999 

13 

92.86 

1 

7.14 

14 

100.00 

-2.94 

A, 000-10, 000 

9 

90.00 

1 

10.00 

10 

100.00 

-2.21 

>  10,000 

10 

83.33 

2 

16.67 

12 

100.00 

-2.02 

Total 

46 

92.00 

4 

8.00 

50 

100.00 

-5.80 

♦All  Z  values  significant  at  the  0.05  level. 


